Acceptability and feasibility of digital adherence technologies for drug-susceptible tuberculosis treatment supervision: A meta-analysis of implementation feedback

Digital adherence technologies (DATs) have emerged as an alternative to directly observed therapy (DOT) for supervisions of tuberculosis (TB) treatment. We conducted a meta-analysis of implementation feedback obtained from people with TB and health care workers (HCWs) involved in TB REACH Wave 6-funded DAT evaluation projects. Projects administered standardized post-implementation surveys based on the Capability, Opportunity, Motivation, Behavior (COM-B) model to people with TB and their health care workers. The surveys included questions on demographics and technology use, Likert scale questions to assess capability, opportunity, and motivation to use DAT and open-ended feedback. We summarized demographic and technology use data descriptively, generated pooled estimates of responses to Likert scale questions within each COM-B category for people with TB and health care workers using random effects models, and performed qualitative analysis of open-ended feedback using a modified framework analysis approach. The analysis included surveys administered to 1290 people with TB and 90 HCWs across 6 TB REACH-funded projects. People with TB and HCWs had an overall positive impression of DATs with pooled estimates between 4·0 to 4·8 out of 5 across COM-B categories. However, 44% of people with TB reported taking TB medications without reporting dosing via DATs and 23% reported missing a dose of medication. Common reasons included problems with electricity, network coverage, and technical issues with the DAT platform. DATs were overall perceived to reduce visits to clinics, decrease cost, increase social support, and decrease workload of HCWs. DATs were acceptable in a wide variety of settings. However, there were challenges related to the feasibility of using current DAT platforms. Implementation efforts should concentrate on ensuring access, anticipating, and addressing technical challenges, and minimizing additional cost to people with TB.


Introduction
Despite the availability of effective treatment, tuberculosis (TB) remains a leading cause of death from an infectious agent [1]. Successful treatment of drug-susceptible TB (DS-TB) requires high levels of adherence to treatment lasting at least 4 months. Since the 1990s, directly observed therapy (DOT) has been employed globally to monitor and support adherence to TB medications. However, there is limited evidence that DOT improves treatment completion in comparison to self-administered therapy [2,3], and treatment completion rates remain below the 90% target in most high burden countries [1]. DOT is associated with catastrophic costs and rigid enforcement of DOT may conflict with the autonomy, dignity, and integrity of people with TB [4,5].
There has been increasing interest in digital adherence technologies (DATs) as a more person-centered alternative to DOT. DATs enable people with TB to take TB medicines at a time and place of their choosing and allow health care workers (HCWs) to monitor adherence and provide tailored adherence support to those who need it [6]. Several different commercial DAT platforms have been deployed in high TB burden countries. These include 99DOTS, a phone-based technology by which people self-report adherence via calls or texts to phone numbers that are revealed behind pills in a medication sleeve [7]; electronic smart pillboxes made of either plastic or cardboard, such as evriMED, which passively register dosing events when the pillbox is opened [8]; and video directly observed therapy (VDOT), whereby people record and upload videos documenting medication ingestion using a smartphone or computer with a camera [9]. Although these technologies are increasingly being implemented, there are limited data on their effectiveness, acceptability, and feasibility, resulting in only a conditional recommendation for their use in TB treatment by the World Health Organization (WHO) [10].
To assess the acceptability and feasibility of DATs from a variety of settings and target populations, the Stop TB Partnership's TB REACH initiative funded 14 DAT projects in 12 countries as part of their Wave 6 funding portfolio [11]. We analyzed standardized implementation feedback obtained from people with TB and HCWs as part of the TB REACH-funded projects.

Studies and participants
We excluded TB REACH Wave 6-funded DAT evaluation projects that did not use a commercial DAT platform, share individual participant data with the Stop TB Partnership or exclusively enrolled people with drug-resistant TB (DR-TB). In addition, we excluded people with TB from the analysis if they were younger than 15 years old or were missing data on sex and excluded HCWs from the analysis if they did not provide direct care to people on TB treatment.

Survey design
Standardized post-implementation surveys for people with TB and HCWs were designed by the TB REACH DAT working group to assess the feasibility and acceptability of using a DAT as part of TB treatment. Feasibility was defined as the extent to which DAT could be successfully used or carried out within a given setting [12]. Acceptability was defined as the perception that the DAT was agreeable, palatable, or satisfactory [13]. To assess feasibility and acceptability, the survey included questions about access to technology and experience using DATs and Likert scale questions, developed to reflect selected domains of the Theoretical Domains Framework. (TDF) [14]. The TDF integrates 33 psychologic theories relevant to behavior change into 14 domains related to Capability (knowledge, attention, memory, and decision processes necessary to use DAT), Opportunity (social and environmental context conducive to using DAT), or Motivation (optimism, reinforcement, and emotion to want to use DAT) to perform a Behavior (COM-B) [15,16]. Health worker surveys also included constructs from the Unified Theory of Acceptance and Use of Technology (UTAUT), a consolidated framework to explain information systems usage behavior [17][18][19]. In addition, the surveys included open-ended questions to further explore DAT feasibility and acceptability, as well as recommendations for improvement (S1 and S2 Tables).
Projects administered surveys to a random sample of people with TB and HCWs. In some cases, projects modified the survey tools to exclude certain questions or include additional questions. Survey administration varied by project and surveys were conducted either by HCWs or research staff. Questionnaires were translated into the local language as needed. This analysis included only survey questions that were used in common by all projects.

Data analysis
Responses from people with TB and HCWs were analyzed separately and stratified by DAT platform evaluated. Demographics and data on access to and use of technology were summarized descriptively. Responses to Likert scale questions were summarized by project using means and standard deviations. A response of agree or strongly agree reflected a favorable impression of DAT. If the question was asked such that agreement indicated an unfavorable impression of DAT, responses were reversed prior to analysis. Composite scores for capability, opportunity, and motivation were created for each person with TB and HCW by calculating the mean response to Likert scale questions within that category.
Pooled estimates for capability, opportunity, and motivation scores across projects were generated using random effects, sample size weighted models. A sample size weighted model was chosen over an inverse variance weighted model due to lack of variance in survey responses across participants for some projects. Secondary analyses assessed differences by gender and age category (�55 years old vs. <55 years old). Differences in mean capability, opportunity, and motivation scores were calculated by gender and by age category for each project. Pooled estimates of the gender and age category differences in mean scores were then generated using the random effects, sample size weighted model described above. Stata 15 was used for all quantitative analyses.
Qualitative analysis of responses to open ended survey questions was performed using a modified framework analysis approach [20]. We began by sorting responses into frames reflecting the open-ended questions posed in each survey. Content that was common across participants (mentioned by four or more participants) or that was uncommon but offered unique insights was flagged for inclusion as a theme. Themes that emerged across frames were considered cross-cutting themes. The main themes across the data sources were then consolidated and synthesized into chart format and organized as barriers/facilitators to DAT uptake by COM-B category, DAT type, project country and sex. This enabled us to identify similarities and differences in barriers and facilitators to DAT uptake across DAT type, country, and sex.

Setting and study population
Of the 14 TB REACH Wave 6-funded DAT evaluation projects, 8 were excluded (2 exclusively enrolled people with drug-resistant TB, 2 used a locally-developed rather than commercial DAT platform, and 4 did not share individual data from persons on TB treatment or HCWs). The remaining six projects administered surveys to 1,316 people with TB and 92 HCWs using 99DOTS or evriMED for TB treatment supervision. Survey data from 1290 (98�0%) people with TB and 90 (97�8%) HCWs were analyzed (Fig 1).
Details of study setting and DAT implementation for the six included projects are provided in S3 Table. Projects in Uganda, South Africa, Tanzania, and Ukraine were conducted at government-run TB treatment facilities, whereas projects in Bangladesh and the Philippines implemented DATs in private sector settings. Most HCWs treated only people with drug susceptible TB, but HCWs surveyed in Ukraine were treating both people with drug susceptible and drug resistant TB.
Projects implementing 99DOTS provided medication blister packs placed within an envelope that revealed toll-free numbers when medications were pushed out of the blister pack. People with TB were instructed to call or text the toll-free numbers daily to confirm dosing. In Tanzania, although people with TB were not charged for text messages to the 99DOTS platform, they were required to have a daily minimum airtime balance on their phones to place calls or texts. Thus, during study recruitment, those who disclosed not regularly keeping a minimum airtime balance on their phones were excluded from the project as they would not be able to text the 99DOTs platform to confirm dosing. Projects implementing evriMED provided a smart pill box that contained TB medication blister packs. In all projects, HCWs could access and monitor daily adherence data using a mobile phone or web-based application.
In total, 1,017 people with TB using 99DOTS in 4 countries and 273 using evriMED in 2 countries completed surveys from September 2019 to May 2020 (Table 1). Almost half were from Bangladesh (n = 599, 46�4%). Sample sizes for the other five projects ranged from 106 to 200 people with TB. Responses to open-ended survey questions were provided by 306 people with TB in Tanzania and the Philippines using 99DOTS, and 19 in Ukraine using evriMED (S4-S6 Tables). Five projects conducted HCW surveys. In total, 33 HCWs using 99DOTS in 4 countries and 20 HCWs using evriMED in Ukraine completed surveys (Table 2).s Sample sizes per project ranged from 12 to 23 HCWs. Responses to open-ended survey questions were provided by 33 HCWs in Tanzania and the Philippines using 99DOTS, and 20 HCWs in Ukraine using evriMED (S5 Table).

Experience using DATs
Overall, 44�1% (n = 233) of people with TB reported having taken their medication but not reporting it using DAT at least once during treatment (Table 3). This was similar among those using 99DOTS and evriMED (42�8% v. 46�0%). The most common reason for not reporting taken doses were problems with charging the battery of their device, poor network connection, forgetting to use the DAT, and the DAT not working. Only 2 (0�9%) responded that they did not want to use the DAT. About one-fifth (n = 132, 22�9%) of people with TB reported missing a dose of medication. Common reasons included forgetting to take their medication, medication side effect, and being too busy. They usually took less than 2 minutes to take their medication and made 1 to 5 visits a month to a TB clinic. Most (n = 335, 78�3%) lived within an hour of their clinic, however 93 (21�7%) had to travel more than an hour.
All HCWs received data from the DAT and most reviewed data daily (n = 73, 90�1%) through phone messages, mobile phone applications, and/or computers (S8 Table). Adherence was most frequently assessed using the mobile phone application (n = 77, 85�6%). In addition, HCWs reported calling people with TB (n = 56, 62�2%), asking when they refilled medications (n = 51, 56�7%), and talking to their family members (n = 44, 48�9%). Throughout the project, 52�2% (n = 47) of HCWs were not able to assess adherence data at some point. HCWs cited problems with poor network connection (n = 38, 80�9) and the mobile phone application not working (n = 34, 72�3%). Some also experienced electricity outages or limited access to records. No HCW responded that they did not want to use the DAT.
Capability was facilitated by knowledge that the DAT was a tool to help people with TB remember to take their medication and improve their adherence (Table 4). They reported that DATs reminded them to take their medications and facilitated adherence monitoring and support.
"The fact that with the smartbox it's impossible to forget to take medicines. It help me to become healthier."-Female 45-54, Ukraine evriMED Some in the Philippines (mostly men) described forgetting to call/text the platform and confusion about the purpose of 99DOTS: "With 99DOTS, sometimes I just really forget to text. I'd sometimes tell myself I'd do it later, but I keep forgetting"-Male, 30-35, Philippines 99DOTS Overall, HCWs also had a favorable impression of their capability to use DAT (Fig 3). HCWs in Tanzania had exclusively strongly agree responses in the capability domain. Pooled estimates for HCWs using 99DOTS (4�89, 95% CI 4�79-4�98) and evriMED (4�78, 95% CI 4�65-4�90) were both high. HCWs felt that the training they received was relevant, useful, and clear, facilitating their capability to successfully implement DATs. HCWs suggested refresher training on any updates to the device/platform would further improve their capability.

Opportunity to use DAT
Within opportunity, people with TB largely thought that using DAT was acceptable and 1,243 (96.4%) would recommend it to their family members. 1,090 (84.5%) agreed or strongly agreed that DAT helped them feel more connected their HCW, and 1,106 (85.7%) strongly disagreed or disagreed that it took too much time to take TB medicine with DAT. But there were some concerns related to using DAT in front of others as it could result in the disclosure of TB status (Fig 2). Pooled estimates for 99DOTS (4�21, 95% CI 4�11-4�32) and evriMED (4�05, 95% CI 3�86-4�24) were less favorable than for capability, but overall still positive with responses averaging above "agree". There were no differences by gender or age (S3 and S4 Figs). People with TB across all studies reported that DATs saved time and money that they would have otherwise spent traveling to clinic each day to take their TB medication under DOT ( In Tanzania, although patients were not charged for messaging the 99DOTS platform, nearly half of participants indicated lack of access to a phone and phone credit (which was required to send free text messages) as major barriers to using 99DOTS. People using both 99DOTS and evriMED reported issues with charging their phones or boxes. Those using evriMED disliked the design of the box, pointing to the cardboard material and large size. Responses by HCW also showed favorable opportunity to use DATs (Fig 3). HCWs using 99DOTS in Tanzania exclusively had "strongly agree" responses to questions in the opportunity domain. Pooled estimates were high for both 99DOTS (4�22, 95% CI 3�90-4�53) and evriMED (4�78, 95% CI 4�60-4�97).
HCWs endorsed decreased workloads and thought that DATs were more person-centered, relieving some of the financial impact of TB treatment and allowing people with TB to go on long trips and continue working. Several HCWs noted that DATs enabled them to spend more quality time engaging with people with TB who needed their support or to attend to other administrative tasks. Barriers included difficulty calling people with TB if they did not have phone credit and lack of access to phones in Tanzania. Additionally, there were technical issues with the evriMED box not closing tightly or not registering when they took their medication and 99DOTS not registering all text messages from them. A small group of HCWs (primarily from Ukraine) perceived DATs increased their workload by adding more forms and procedures for them to complete each day.

Motivation to use DAT
People with TB were motivated to complete treatment and get healthy, but there were concerns about data privacy (Table 4). Motivation was favorable for using both 99DOTS (pooled estimate 4�52, 95% CI 4�17-4�86) and evriMED (pooled estimate 4�52, 95% CI 4�17-4�86) (Fig 2). There were no differences in pooled estimates of motivation to used 99DOTS by gender or age, but among people with TB using evriMED, mean motivation scores were higher among older adults (�55 vs. <55 years old). (S5 to S6 Figs).

Discussion
Overall, we found that 99DOTS and evriMED were both highly acceptable to people receiving treatment for DS-TB and HCWs caring for them. However, there were barriers encountered that decreased the feasibility of DATs, some of which can be targets for future optimization of these technologies and their implementation. Gender did not have an impact on the perceived acceptability or feasibility of the two DAT platforms; however, motivation to use the evriMED was higher among older adults (>55 years). While it is often thought that digital health tools may be less acceptable to older adults, our finding is similar to other research suggesting that higher age is associated with higher perceived usefulness of digital health tools [21].
Our findings largely support prior studies demonstrating DATs offer an acceptable, person-centered approach to TB care [22][23][24]. Although, projects included in our analyses varied in terms of network infrastructure, access to mobile phones, target population and settings (public v private clinics), overall, high acceptability was demonstrated by high capability, opportunity, and motivation to use 99DOTS and evriMED among people with TB and HCWs in all projects. Both agreed that DATs saved time and money which would have otherwise been spent traveling to clinics. Our study also found that people with TB using DAT felt strongly connected to their HCWs. Most HCWs endorsed reduced workload when using DAT. Thomas et al reported similar findings among HCWs at government-run TB centers in India [25]. Stigma related to taking TB medication contributed to lower opportunity scores, reflecting the known impact of stigma on TB care [26].
In addition to stigma, barriers to the feasibility of implementing DATs included limited access to mobile phones, network connectivity, and technical issues with phones and the two DAT platforms. The projects evaluating 99DOTS required that participants have access to mobile phones, and specifically in the case of Tanzania, a minimum airtime balance, to participate in the projects. The data reported here therefore do not reflect the perspectives of all people with TB. Moreover, these issues highlight the need to ensure access to DATS in an equitable manner. Although daily access to cell phones among people with TB included in the projects was high, 44% reported having taken their TB medication but not reporting it using their DAT. Issues with charging phones/boxes and network connectivity were barriers to using DATs across all projects. In Tanzania, the only 99DOTS project where a toll-free number still required phone credit, nearly half of people with TB completing open-ended questions mentioned the cost associated with needing phone credit as a key barrier. A minority reported other issues with their DAT, such as the evriMED box not closing tightly and the 99DOTS platform not registering texts or calls that were sent from people with TB. Similar technical barriers have been reported previously in China, India, Peru, and sub-Saharan Africa [25,[27][28][29][30].
Our analysis had some limitations. First, each project was unique in its study design, target population, and implementation methods, therefore the survey sampling and administration skills varied across projects making it difficult to ascertain all potential causes of selection bias for the studies included in this meta-analysis. For example, most projects administered their surveys pre-COVID-19, but both Bangladesh and South Africa administered their surveys post-COVID-19 protection measures, potentially resulting in more favorable responses when there was an increased awareness of the important role that digital health tools can offer compared to in-person care. Some projects required participants to have regular access to a phone and phone airtime credit, which may have prevented more socially or economically disadvantaged people with TB from participating in the intervention and survey. More than half of the people with TB responding to the 99DOTS survey came from Bangladesh, so their responses are overrepresented in pooled estimates; however mean capability, opportunity, and motivation scores from Bangladesh fell near the median of other project responses, indicating that this was unlikely to have a large impact on the results. Similarly, for evriMED, only HCWs from Ukraine completed the survey. Therefore, we are unable to pool results from multiple projects and these responses may not reflect the experience in other geographic areas. Second, since the surveys were implemented by the project teams, participating people with TB and HCWs may have felt pressure to please the person administering the survey. Social desirability bias may result in more favorable responses. Third, selection of DAT was not randomized, which prevented direct comparison of acceptability and feasibility of the two DAT platforms; differences observed may reflect differences in setting or project implementation. Finally, qualitative data was only collected via open-ended survey questions and not through in-depth interviews. This may have prevented us from gaining a deeper and more nuanced understanding of the barriers and facilitators to DAT implementation and acceptability. There are other commonly used DATs not represented in these data. Similar findings for those using video observed therapy, another DAT, were noted in other studies [31].
In summary, the available data show high acceptability of DATs and support further scaleup of DAT for treatment of DS-TB. DATs can be an acceptable, person-centered care option for many people with TB. They have the potential to address some of the structural challenges (access to care, time, distance, and costs) many people with TB face. However, for many underserved TB populations, these solutions may still be out of reach with the existing digital divide in many countries, particularly among rural and poorer populations. Without access, digital health solutions can potentially compound health inequities and further marginalize these populations. Thus, implementation efforts should focus not only on anticipating and addressing technical challenges with the platforms and minimizing additional costs to people with TB; the focus should also be on ensuring access and digital health equity.
Supporting information S1